The field of the invention relates to video processing and more particularly to the altering of video images.
It is a common practice for a director to provide the xe2x80x9cmost advantageous viewxe2x80x9d of the subject matter. Sporting events, for example, usually have multiple cameras strategically positioned around the event to capture the most important point-of-view that changes position over time. Changing this view point can be as simple as the director switching from one camera""s output to another camera""s output, or having the camera operator re-position or re-orient the camera to a new setting. Similarly, a newsworthy event requires the best vantage point to best capture the moment and gather all of the pertinent happenings around the event.
In many cases, it is difficult to capture the best view of the subject matter. This may be due to the difficulty of camera placement (behind a group of other reporters who swarm a celebrity), or not switching quickly enough between cameras (the director makes a human error). To overcome the problem of camera placement, many techniques has been devised. One solution is to mount remote cameras onto various objects, such as on a remote control aircraft, placement on a participant""s body (in a quarterback""s football helmet), mounting the camera on a pole held by the operator, etc.
In order to solve the possible director error, and to also add value to the product presented to the viewer, multiple video feeds can be sent to the viewer, who can select their favorite vantage point, or see multiple vantage points at one time. New data streaming standards, such as ASF and MPEG-4, for example, are planned to allow multiple video feeds through a single compressed data stream for this purpose, among other attributes. Other ingenious methods include sending a wide-area view, or surround-view, from a single camera, or the use of IPIX (e.g., U.S. Pat. Nos. 5,185,667, 5,313,306, 5,359,363, 5,764,276), which sends a warped view, but is de-warped at each viewer""s discretion.
While prior art methods are effective, they rely on a certain level of decision-making of a producer. Accordingly, a need exists for a method of video processing which is less dependent upon the judgment of a video producer.
Under one illustrated embodiment of this invention multiple video feeds are taken and used to create a new, user controlled vantage point between the multiple video feeds. The effect of this invention is to create new vantage points as if there was a camera that could float between the positions of the other cameras.
This invention provides a novel system for processing multiple video streams. Multiple video cameras provide different angular views of the same subject matter. Knowing the position and orientation of each camera relative to each other or from a defined reference, the video streams are combined using three-dimensionally correct perspective morphing to provide a new video output stream that is an interpolated view between the cameras"" views. The interpolated position of the output video is determined by an input that allows any number of views that include or fall between the multiple video streams.
Under illustrated embodiments discussed below, a method and apparatus are provided for synthesizing an image from a plurality of video streams. The method includes the steps of determining an orientation of a first and second video source of the plurality of video streams with respect to a video object lying within a field of view of both the first and second video sources and selecting an orientation of a synthesized video source with respect to the video object. The method further includes the step of interpolating among corresponding images of the first and second video sources to form a stream of perspective images of the video object from the selected orientation of the synthesized source.